Goto

Collaborating Authors

 Grodno Region


Recurrence Meets Transformers for Universal Multimodal Retrieval

Caffagni, Davide, Sarto, Sara, Cornia, Marcella, Baraldi, Lorenzo, Cucchiara, Rita

arXiv.org Artificial Intelligence

With the rapid advancement of multimodal retrieval and its application in LLMs and multimodal LLMs, increasingly complex retrieval tasks have emerged. Existing methods predominantly rely on task-specific fine-tuning of vision-language models and are limited to single-modality queries or documents. In this paper, we propose ReT-2, a unified retrieval model that supports multimodal queries, composed of both images and text, and searches across multimodal document collections where text and images coexist. ReT-2 leverages multi-layer representations and a recurrent Transformer architecture with LSTM-inspired gating mechanisms to dynamically integrate information across layers and modalities, capturing fine-grained visual and textual details. We evaluate ReT-2 on the challenging M2KR and M-BEIR benchmarks across different retrieval configurations. Results demonstrate that ReT-2 consistently achieves state-of-the-art performance across diverse settings, while offering faster inference and reduced memory usage compared to prior approaches. When integrated into retrieval-augmented generation pipelines, ReT-2 also improves downstream performance on Encyclopedic-VQA and InfoSeek datasets. Our source code and trained models are publicly available at: https://github.com/aimagelab/ReT-2


Evaluating Inter-Column Logical Relationships in Synthetic Tabular Data Generation

Long, Yunbo, Xu, Liming, Brintrup, Alexandra

arXiv.org Artificial Intelligence

To evaluate the fidelity of synthetic tabular data, numerous metrics have been proposed to assess accuracy and diversity, including both low-order statistics (e.g., Density Estimation and Correlation Score (Zhang et al., 2023), Average Coverage Scores (Zein & Urvoy, 2022)) and high-order statistics (e.g., α-Precision and β-Recall (Alaa et al., 2022)). However, these metrics operate at a high level and fail to evaluate whether synthetic data preserves logical relationships, such as hierarchical or semantic dependencies between features. This highlights the need for a more fine-grained, context-aware evaluation of multivariate dependencies. To address this, we propose three evaluation metrics: Hierarchical Consistency Score (HCS), Multivariate Dependency Index (MDI), and Distributional Similarity Index (DSI). To assess the effectiveness of these metrics in quantifying inter-column relationships, we select five representative tabular data generation methods from different categories for evaluation. Their performance is measured using both existing and our proposed metrics on a real-world dataset rich in logical consistency and dependency constraints. Experimental results validate the effectiveness of our proposed metrics and reveal the limitations of existing approaches in preserving logical relationships in synthetic tabular data. Additionally, we discuss potential pathways to better capture logical constraints within joint distributions, paying the way for future advancements in synthetic tabular data generation.


A set of semantic data flow diagrams and its security analysis based on ontologies and knowledge graphs

Brazhuk, Andrei

arXiv.org Artificial Intelligence

For a long time threat modeling was treated as a manual, complicated process. However modern agile development methodologies and cloud computing technologies require adding automatic threat modeling approaches. This work considers two challenges: creating a set of machine-readable data flow diagrams that represent real cloud based applications; and usage domain specific knowledge for automatic analysis of the security aspects of such applications. The set of 180 semantic diagrams (ontologies and knowledge graphs) is created based on cloud configurations (Docker Compose); the set includes a manual taxonomy that allows to define the design and functional aspects of the web based and data processing applications; the set can be used for various research in the threat modeling field. This work also evaluates how ontologies and knowledge graphs can be used to automatically recognize patterns (mapped to security threats) in diagrams. A pattern represents features of a diagram in form of a request to a knowledge base, what enables its recognition in a semantic representation of a diagram. In an experiment four groups of the patterns are created (web applications, data processing, network, and docker specific), and the diagrams are examined by the patterns. Automatic results, received for the web applications and data processing patterns, are compared with the manual taxonomy in order to study challenges of automatic threat modeling. Keywords: threat modeling, data flow diagram, ontologies, knowledge graph, OWL, RDF, SPARQL, DFD.


Towards automation of threat modeling based on a semantic model of attack patterns and weaknesses

Brazhuk, Andrei

arXiv.org Artificial Intelligence

This works considers challenges of building and usage a formal knowledge base (model), which unites the ATT&CK, CAPEC, CWE, CVE security enumerations. The proposed model can be used to learn relations between attack techniques, attack pattern, weaknesses, and vulnerabilities in order to build various threat landscapes, in particular, for threat modeling. The model is created as an ontology with freely available datasets in the OWL and RDF formats. The use of ontologies is an alternative of structural and graph based approaches to integrate the security enumerations. In this work we consider an approach of threat modeling with the data components of ATT&CK based on the knowledge base and an ontology driven threat modeling framework. Also, some evaluations are made, how it can be possible to use the ontological approach of threat modeling and which challenges this can be faced.


Multilevel regression with poststratification for the national level Viber/Street poll on the 2020 presidential election in Belarus

Zahorski, Ales

arXiv.org Machine Learning

Independent sociological polls are forbidden in Belarus. Online polls performed without sound scientific rigour do not yield representative results. Yet, both inside and outside Belarus it is of great importance to obtain precise estimates of the ratings of all candidates. These ratings could function as reliable proxies for the election's outcomes. We conduct an independent poll based on the combination of the data collected via Viber and on the streets of Belarus. The Viber and the street data samples consist of almost 45000 and 1150 unique observations respectively. Bayesian regressions with poststratification were build to estimate ratings of the candidates and rates of early voting turnout for the population as a whole and within various focus subgroups. We show that both the officially announced results of the election and early voting rates are highly improbable. With a probability of at least 95%, Sviatlana Tikhanouskaya's rating lies between 75% and 80%, whereas Aliaksandr Lukashenka's rating lies between 13% and 18% and early voting rate predicted by the method ranges from 9% to 13% of those who took part in the election. These results contradict the officially announced outcomes, which are 10.12%, 80.11%, and 49.54% respectively and lie far outside even the 99.9% credible intervals predicted by our model. The only marginal groups of people where the upper bounds of the 99.9% credible intervals of the rating of Lukashenka are above 50% are people older than 60 and uneducated people. For all other marginal subgroups, including rural residents, even the upper bounds of 99.9% credible intervals for Lukashenka are far below 50%. The same is true for the population as a whole. Thus, with a probability of at least 99.9% Lukashenka could not have had enough electoral support to win the 2020 presidential election in Belarus.


Artificial intelligence in Health Insurance - Current Applications and Trends

#artificialintelligence

Health insurance is a critical component of the healthcare industry with private health insurance expenditures alone estimated at $1.1 billion in 2016, according to the latest data available from the Centers for Medicare and Medicaid Services. This figure represents 34 percent of the 2016 National Health Expenditure at $3.3 trillion. In this article, we will look at four AI applications that are tackling problems of underutilization and fraud in the insurance industry. Some applications below claim that they are using artificial intelligence to help improve health insurance cost efficiency, while reducing waste of money on underutilized or preventable care. Other applications claim to detect fraudulent claims.